Offline location-based consumer metrics using online signals

ABSTRACT

A business monitoring system is described herein that brings together the previously separate worlds of social media and offline secret shopper and similar programs. With the business monitoring system, owners of brands are able to monitor the local voice of the customer to detect local and regional trends in sentiment and activity, build benchmarks and goals for local storefronts, evaluate in-store operations and customer service trends, and measure the local impact of marketing and advertising initiatives. The system collects and analyzes signals from online sources, producing reports, analytics, benchmarks, and alerts regarding offline activity at the local/store-front level. The system normalizes the signals from various sources, analyzes the signals at the individual location level, aggregates the data across various dimensions, builds benchmarks for comparison, and fires triggers notifying appropriate people upon detecting a meaningful variance. Thus, the system provides a rich and timely set of information to business decision makers.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 13/573,783, filed on Oct. 3, 2012, which claims the benefit ofU.S. Provisional Patent Application No. 61/542,460, filed on Oct. 3,2011, each of which are hereby incorporated by reference.

BACKGROUND

Metrics are very important to the operation of a business. Businessesuse metrics to plan future expansion, assess customer interest inproducts, receive feedback about areas for improvement, and so forth.Metrics may include sales information (e.g., for specifics products andspecific locations), customer opinions, product volume, inventory, andso on. Traditional brick and mortar businesses collect metrics throughtheir own unique processes depending on how they track sales. Forexample, some accounting systems provide metrics, while some businessesmay dedicate whole departments to gathering and tracking metrics-relatedinformation. Online businesses, such as e-commerce websites, benefitfrom an inherent collection of metrics. Because their visitors and salesoccur electronically, a good trail of the consumer's behavior andactions is available.

Secret shoppers are hired by many stores with physical locations to testthe customer experience at those locations. A secret shopper is a surveyrespondent paid to go to a store, shop there, and report to the companyor a third party about various aspects of the shopping experience.Stores may test how locations are treating customers, how locations arecomplying with regulations, whether particular rare conditions (e.g., apeanut allergy at a restaurant) were handled appropriately by staff, andso on. Secret shoppers are very useful for location-specific analysis,but they may fail to capture the experience of every customer andcustomer type, and the process misses the types of places (typicallyonline) where customers of a business share opinions today.

A recent study indicates that consumer brands with brick and mortarlocations missed more than 70% of local customer feedback. This “localblind spot” is the direct result of the exploding consumer adoption ofmobile, social, and location-based services. Many brands using leadingsocial media and brand monitoring solutions do not realize they havethis blind spot. The study found that keyword-based social mediamonitoring solutions, while an important piece of an overall strategy,failed to comprehensively surface in-store consumer content. Forinstance, keyword-based monitoring for Costco missed 63% of localconsumer feedback at their warehouses. The root cause of the blind spotis that most solutions in the market today are designed for broadmonitoring across the Internet, using keyword and key phrase matching.In addition, Internet-wide monitoring is not particularly good atsorting out specific information related to individual locations of amulti-location business.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates components of the businessmonitoring system, in one embodiment.

FIG. 2 is a flow diagram that illustrates processing of the businessmonitoring system to build metrics by tracking online occurrences andinferring offline behavior from those occurrences, in one embodiment.

FIG. 3 is a flow diagram that illustrates processing of the businessmonitoring system to receive a request to monitor a brand's performanceat a physical location through online events, in one embodiment.

FIG. 4 is a display diagram that illustrates a metric reportingdashboard of the business monitoring system, in one embodiment.

FIG. 5 is a display diagram that illustrates four widgets for displayingbusiness metrics, in one embodiment.

FIG. 6 is a display diagram that illustrates two more widgets fordisplaying business metrics, in one embodiment.

FIG. 7 is a display diagram that shows a map-based display of metrics,in one embodiment.

DETAILED DESCRIPTION

A business monitoring system is described herein that is based on abottom up architecture, ensuring that local content explicitly tied to abrand location surfaces—regardless of a match to a pre-determined set ofkeywords. The system brings together the previously separate worlds ofsocial media and offline secret shopper and similar programs. With thebusiness monitoring system, owners of brands are able to monitor thelocal voice of the customer to detect local and regional trends insentiment and activity, build benchmarks and goals for localstorefronts, evaluate in-store operations and customer service trends,and measure the local impact of marketing and advertising initiatives.The system collects and analyzes signals from online sources, producingreports, analytics, benchmarks, and alerts regarding offline activity atthe local/store-front level. The system normalizes the signals fromvarious sources (reviews, check-ins, mentions, and so on), analyzes thesignals at the individual location level, aggregates the data acrossvarious dimensions, builds benchmarks for comparison, and fires triggersnotifying appropriate people upon detecting a meaningful variance. Theresulting system produces alerts, trends, and analyses foroffline/in-store location-based activity, using online signals.Comparisons between trends and analyses produced by the system havecorrelated meaningfully with offline verification. Thus, the businessmonitoring system provides a rich and timely set of information tobusiness decision makers.

As used herein, location may refer to any specific venue or physicaladdress, perhaps within a larger address (e.g., a storefront within amall, or a concession stand at a baseball stadium). A site refers to anonline website or service where people talk about offline activity(e.g., Facebook, Twitter, online forums, and so on). A source refers toa particular page or venue within a site talking about a particularlocation (e.g., “The White House's Facebook Place page”). A consumer orsource member is a person who interacts with a source. A brand is a setof locations with some associated metadata. A signal is any contentretrieved or recorded from a source, including check-ins, reviews,mentions, and the like. A signal also includes the change in consumercount (e.g., added/removed a fan/follower).

The information provided by the business monitoring system can be usedfor a wide variety of applications relevant to improving businessoperations. A few of these applications are provided in the followingparagraphs by way of example, including monitoring effectiveness ofonline and offline marketing campaigns, detecting in-store operationalproblems, identifying storefront innovation, producing competitiveintelligence, and inferring store groupings based on implicit customerprofiles.

The first example described herein is using the business monitoringsystem to monitor the effectiveness of online and offline marketingcampaigns. Staff can instruct the system to time-box a set of locations,producing an overall view of a marketing campaign, measuring the signalscaptured—reflecting both online and offline results. The view includesboth overall change metrics (new people engaged with the trackedlocations during the period, total amount of activity/signal datagenerated at the tracked locations during the period, and so on) andtrends within the campaign (day by day or week by week performancewithin the locations tracked). An example is the measurement of theeffectiveness of a new billboard advertisement placed near a storelocation (or in the same city, for example). The system gathers onlinesignals and can compare the consumer sentiment towards the businessbefore and after the billboard goes up. The changes measured aresufficiently granular and can be compared with changes at otherlocations (e.g., a control group) to determine an amount of change thatcan be inferred to result from the billboard. Thus, online signalsprovide data of the effectiveness of offline advertising.

The campaign can be compared both to historical and concurrentbenchmarks. For instance, if the campaign tracks five locations foreight weeks, the system can compare the campaign to the metrics of thosesame five locations for the eight weeks prior to the campaign start, aswell as to the metrics of the other locations within the brand duringthose same eight weeks. The system can weigh the concurrent benchmarkcomparison by location count to produce comparative metrics. If the fivelocations added 100 followers and the other 10 locations added 50followers, the five locations generated four times more followers thanthe other locations (the five locations generated 20 followers perlocation, while the 10 generated only five per location), and thischange can be appropriately correlated with the campaign.

Another example is using the business monitoring system to detectin-store operational problems. Notifications provide a way for therecipient to be informed when a set of locations begin generating anabnormally high number of complaints, and expose the trending topics andcomplaints themselves to the recipient. These notifications in turnoften include specific customer feedback about staff, facilities,service, or other operational issues. The recipient of the notificationcan then work with the customer to resolve the complaint, and work withthe locations involved to address the underlying operational problem.Other metrics may also contribute to a report of operational problems,such as a sales drop at one location compared to other comparablelocations during the same period.

Another example is using the business monitoring system to identifystorefront innovation. Through campaign tracking, notification, orinteractive review, locations that are performing significantly betterthan other locations at certain metrics can be highlighted graphically.The person reviewing the highlighted performance can drill into theactivity reflected in the metrics, and determine whether there areparticular topics or messages that encouraged the improved customerbehavior. Additionally, the person can reach out to the staffresponsible for the particular improved locations to request detail asto what other efforts they have taken to improve the performance oftheir locations. This can assist in improving other locations.

The business monitoring system can also be used to produce competitiveintelligence. The system can generate specific benchmarks for thelocations of competitive brands, tracking their publicly available data.Those benchmarks can be used to compare the brand to its competitors ona brand-wide, segment by segment, or location basis, using the metricstracked as part of the system.

The business monitoring system can also be used to infer store groupingsbased on implicit customer profiles. Each of the profiles that left thesignals on the sources are collected and themselves analyzed, generatingseveral quantitative dimensions. One dimension may include a number ofinteractions overall, grouped by type (check-in, review, mention, and soon), grouped by sentiment (positive vs. neutral vs. negative). Anotherdimension may include a number of interactions containing words from aseries of word lists (e.g., a word list containing swears, anothercontaining brand names or trademarks, and so on). There can existstandardized word lists used for all brands, as well as brand-specificword lists.

The resulting dimensions have a single number for the profile for eachword list (“this person left feedback mentioning a brand name threetimes”), as well as series of numbers, one per word within each wordlist (“this person left feedback mentioning ‘big mac’ two times”).Another dimension is demographic data, either inferred or explicit—agerange, gender, education level, and hometown. Another dimension islocation data—quantifying the number of times the person interacted withan individual location, with a location in a town/city, with a locationin a state/region. Another location is temporal data—quantifying howoften the person interacts during the day vs. during the night, as wellas weekday vs. weekend, and season. Other dimensions may includeadditional quantitative data extracted from the profile metadata andinteraction history.

In some embodiments, the qualitative profile dimensions are fed into asparse matrix and clustered with a vector clustering process, such asbisecting K-Means, producing 50+ clusters. The locations within thebrand are then grouped according to how often profiles from each of theclusters interact with them. The resulting grouping of locations canthen be used to target marketing campaigns (e.g., “people like *this* goto these locations, so run a targeted campaign there”), define customlocation segments for benchmarking (e.g., “this location is performingsignificantly better [or worse] than other locations with similarcustomers”).

FIG. 1 is a block diagram that illustrates components of the businessmonitoring system, in one embodiment. The system 100 includes amonitoring data store 110, monitoring user interface 120, searchcomponent 130, source identification component 140, signal acquisitioncomponent 150, signal analysis component 160, data aggregation component170, benchmark building component 180, variance detection component 190,and notification component 195. Each of these components is described infurther detail herein.

The monitoring data store 110 stores data collected and analyzed by thesystem 100 to build metrics for reporting to one or more reportrecipients. The report recipients may include the brand owners,competitors, service providers, or any other party interested in abusiness. The monitoring data store 110 may include one or more files,file systems, hard drives, databases, cloud-based storage services, orany other facility for storing data. From an interactive perspective,the data store 110 may be partitioned and queried independently on abrand basis. For benchmarking/non-interactive purposes, cross-brandqueries are supported, though interactivity is not required. Themonitoring data store 110 builds a historical log of data collected overtime, so that comparisons can be made to detect relevant events and tosuggest action changes or marketing improvements.

The monitoring data store 110 includes location data that provides acollection of geographic points (“locations”), grouped together into“brands”, representing the locations to be tracked. The location datastore also includes brand-specific categorization of each location(e.g., “store type” of “kiosk” vs. “store type” of “stand-alone”), andreferences to the sources from which each location can acquire localsignal data. The monitoring data store 110 also contains data acquiredfrom each signal. Data from each signal is retrieved and analyzed, withsignal data stored by “source”, which in turn is tied to zero or more“locations”, and tied to one “brand”. The content may include cacheddata and derived metadata to categorize the signal data.

The monitoring data store 110 also contains benchmark information.Brands and locations themselves have metadata tied to them—bothimplicitly defined (e.g., geographic, demographic, industryclassification, brand/location type) and explicitly defined (e.g.,brand-determined categorizations). Signal data is stored in aggregateaccording to the benchmark dimensions available, tied to a timecomponent to track changes over time (e.g., to record the fact thatcoffee shops in Seattle had on average X new visitors in July 2011, or Ycustomer interactions on Aug. 3, 2011).

The monitoring user interface 120 receives input from report recipientsto define the type of monitoring information to be tracked and outputsreports to report recipients that contain the requested monitoringinformation. The user interface 120 includes a facility to definelocations and sources within a brand, specify triggers for alerting (towhom and when), and to query aggregate brand data interactively. Theinterface 120 may include a web interface with web-based forms,interactive maps and graphs, downloadable spreadsheets, and emailsdelivered to notify recipients of triggering activity. The interface mayalso include one or more mobile applications (e.g., for a smartphone ortablet), desktop applications, and a programmatic interface (e.g., a webservice or object model) through which other services can leverage thepower of the business monitoring system 100 to provide extendedservices.

The search component 130 provides query-based access to the monitoringdata store 110 for answering business inquiries. In some embodiments,the data displayed is a set of rollup values stored in variousdatabases, but for ad-hoc queries and content drill-down, the content isindexed in an inverted index system (e.g., Solr/Lucene), exposing a highperformance query interface with faceting on location, source type,interaction type, sentiment, segment, and demographic data. The searchcomponent 130 allows business owners or others to extract data along anyaxis that interests them, and can provide answers to specific questions(e.g., what products are buyers near store X interested in, how effectof the advertisements placed in region Y, and so forth).

The source identification component 140 identifies signal sourcesrelevant for a particular location and brand. In some embodiments, eachlocation is coded with latitude and longitude, as well as the location'sname (a local nickname, trade name, or the brand's name). For eachlocation, each of the integrated source sites is queried to identify thesource page for that location's signals. The manner of querying dependson the source site itself (e.g., through a Foursquare venue searchapplication-programming interface (API), through a third party databasesuch as Locationary or Factual, or through a third party searchengine/index such as Google with site search filters).

In some embodiments, the system periodically tries to fill in gapsthrough crowdsourcing the source discovery. For example, the system maycreate Mechanical Turk Human Intelligence Tasks (HITs) that request thatMechanical Turk workers look at a search results page and pick thesource associated with the business/location. These tasks may show thehigh-level criteria (location name, address, and site), display thesearch results page for that site's search, and then pay out a smallamount (e.g., $0.04) to have the worker enter the correct source'suniform resource locator (URL). The system verifies sources by checkingtheir general accuracy (e.g., are they for a supported site?). Thesystem associates the sources with the location provided, with animplicitly determined location within the brand (e.g., by parsingaddress information from the source data), or with the brand as a whole.Clients/brands and support staff periodically review the sources foundand their assignment to locations to provide another layer ofvalidation. Sources can be trivially disabled or reassigned to anotherlocation, as needed.

The signal acquisition component 150 acquires new data from specifiedsignal sources. A set of sites are identified and integrations developedfor each, designed to acquire signals through the site's API or webinterface while complying with appropriate terms of use and querylimitations. Sites are selected as they increase in prominence forlocation-based consumer activity, and may include Foursquare, Gowalla,Facebook Places, Facebook, Twitter, Yelp, CitySearch, InsiderPages,Citygrid, Judysbook, Yahoo Local, Google Places, and others.Integrations use a site-provided API where available and appropriate,but may also use standard web scraping/crawling tools to parse availablemetadata. The component 150 queries sites for their sources periodically(e.g., daily). The query rate may vary based on the terms of service forthe individual sites, performance, or other factors. When queried,various types of signals are acquired, such as total number of consumerswho have interacted with the source, total number of interactions todate, content those consumers have shared, ratings or metadataassociated with individual interactions, as well as metadata about theindividual source and location itself.

The signal analysis component 160 analyzes data received from eachsource to identify brand and location relevant information. Each pieceof content is analyzed for both structural and textual meaning, as wellas for demographic metadata about the person who left the content. Insome embodiments, content analysis is performed asynchronously whenevernew data is acquired. Structural data includes the name of the personwho left the signal, any picture associated with their profile, thedate/time that the data was left, and any explicit scores or topicsentered. Textual analysis may be performed using standard search enginetechnology (e.g., Lucene/Solr) to identify the topics and phrases withinthe data.

In some embodiments, semantic analysis is performed on the textualsignal left, extracting both high-level mood estimates and additionalnormalized topics. Semantic analysis can be performed with standardoff-the-shelf text analysis engines, such as OpenAmplify's web service,or via proprietary solutions. In some embodiments, demographic analysisis performed on the textual signal left as well, augmenting anyexplicitly provided demographic data about the author who left thecontent (e.g., age, gender, education level). When explicit demographicdata is not available, implicit demographic data can be estimated usingstatistically derived text models, powered by models such as locallytrained Bayesian classifiers or a web service such as uClassify.

The data aggregation component 170 aggregates data to discover trendsand useful information across retrieved data points. On a periodic(e.g., day-by-day) basis, the component 170 compactly summarizes thestate of individual sources into useful statistics, such as: totalnumber of people that have followed or interacted with the source, totalnumber of people that have a bidirectional relationship with the source,and total number of check-ins, reviews, and messages that have been leftat the source. In some embodiments, the source summaries are aggregatedinto location summaries, combining the data in each location's sourcesinto useful statistics by day (or other grouping), including: newmessages, check-ins, and reviews added that day, number of new messagesthat are overall positive in nature, number of new messages that areoverall negative in nature, number of new messages that are overallneutral in nature, total number of messages received, new followersadded that day (unidirectional relationship), and new fans added thatday (bidirectional relationship).

In some embodiments, the data aggregation component 170 then groupslocation aggregation by brand-defined dimensions, such as store type,state, region, and department. The locations associated with each of thesegments in each dimension are rolled up into a segment summary with afew useful statistics by period. The rollup is an aggregation with eachrecord showing data such as the total new fans on a particular day forall “kiosk” locations or the total number of new positive signals forall locations managed by a particular person.

The benchmark building component 180 builds relevant benchmarks fortracking brand information. Benchmarks are both intra-brand andsystem-wide. Intra-brand benchmarks are built from the historical trendsof individual sources, locations, and segments. Inter-brand benchmarksare built by using the additional metadata tied to locations and brandsto determine which locations and segments should be compared againsteach other (e.g., coffee shops to coffee shops, rural shops to ruralshops instead of rural shops to urban shops, and so on). In someembodiments, intra-brand benchmarks are built implicitly by persistingand querying the aggregate analysis data.

In some embodiments, inter-brand benchmarks are built both explicitlyand on an ad-hoc basis. They are built by using the location and brandmetadata to define a particular type of benchmark (e.g., industrybenchmark), the function to classify which locations belong in eachcategory (e.g., Standard Industrial Classification (SIC)/North AmericanIndustry Classification System (NAICS) code), and then aggregating theunderlying location summaries accordingly. Benchmarks can also be brokendown with additional categorization (e.g., an industry plus populationbenchmark), which uses more complex classification functions (e.g.,SIC/NAICS code plus Urban/Suburban/Rural).

In some embodiments, benchmarks are built according to actual values, aswell as with derivative functions, such as percent growth per capita forthe town/city the location is in, percent growth per online communitymember, and percent growth per source site community size (e.g., totalnumber of Twitter users, total number of Foursquare users). Benchmarksmay be persisted and queried in the same way that brands as a whole are,using dimensions and segments to store the aggregate data thensnapshotted using summary data.

The variance detection component 190 detects defined occurrences in thedata that indicate anomalies or other events of interest. Each brand candefine a set of triggers, specifying who should be notified when certainattributes exceed the normal parameters. In some embodiments, triggersare defined within the brand at the individual location level, at thesegment level, at the brand level, or at some combination thereof.Triggers can be defined to be evaluated on a daily, weekly, or otherbasis. When a trigger is due to be evaluated, the locations beingcovered are compared across the defined aspects to the appropriatebenchmark, and if the data varies more than expected, the trigger isfired, producing a notification.

Triggers can be defined on any of the tracked metrics, such as peopleengaged with the store, positive or negative mentions, number ofinteractions or reviews, and so forth. Triggers can be tracked againstthe associated locations' historical trends as a benchmark (e.g., “didthese stores have significantly more or less check-ins this week thanthey usually did over the last 6-12 months?”), or against anotherbenchmark in the system (e.g., “did these stores have significantly moreor less check-ins this week than other coffee shops had?”). Triggers maydefine the threshold for notification (e.g., “>5% above the average”),use the standard deviation as the implicit threshold (e.g., “it usuallyvaries −5 to +5%, but if it varies by 8%, fire the trigger”), or applyother criteria.

The notification component 195 provides a notification based on detectedevents from the variance detection component 190. The notificationcomponent may use email, text messaging, push notifications to a mobiledevice, or any other facility for communicating with users configured tobe notified when events occur. The notification component 195 mayinclude information in the notification such as a textual description ofthe event that led to the notification, a severity of the event, and soforth. The notification component 195 may be configured to notifymultiple people in relation to the same event (e.g., a store manager anddistrict manager).

The computing device on which the business monitoring system isimplemented may include a central processing unit, memory, input devices(e.g., keyboard and pointing devices), output devices (e.g., displaydevices), and storage devices (e.g., disk drives or other non-volatilestorage media). The memory and storage devices are computer-readablestorage media that may be encoded with computer-executable instructions(e.g., software) that implement or enable the system. In addition, thedata structures and message structures may be stored oncomputer-readable storage media. Any computer-readable media claimedherein include only those media falling within statutorily patentablecategories. The system may also include one or more communication linksover which data can be transmitted. Various communication links may beused, such as the Internet, a local area network, a wide area network, apoint-to-point dial-up connection, a cell phone network, and so on.

Embodiments of the system may be implemented in various operatingenvironments that include personal computers, server computers, handheldor laptop devices, multiprocessor systems, microprocessor-based systems,programmable consumer electronics, digital cameras, network PCs,minicomputers, mainframe computers, distributed computing environmentsthat include any of the above systems or devices, set top boxes, systemson a chip (SOCs), and so on. The computer systems may be cell phones,personal digital assistants, smart phones, personal computers,programmable consumer electronics, digital cameras, and so on.

The system may be described in the general context ofcomputer-executable instructions, such as program modules, executed byone or more computers or other devices. Generally, program modulesinclude routines, programs, objects, components, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Typically, the functionality of the program modules may becombined or distributed as desired in various embodiments.

FIG. 2 is a flow diagram that illustrates processing of the businessmonitoring system to build metrics by tracking online occurrences andinferring offline behavior from those occurrences, in one embodiment.

Beginning in block 210, the system receives a list of one or moresources that provide online signals relevant to a particular physicallocation and brand. The system may access the list of sources from adata store where the source list was stored after a brand owner or otherperson configured the system to track metrics for the physical locationand brand. The sources may include a variety of online locations,specified by URL or other identifier, where consumers of the physicallocation and brand congregate online and discuss the businessrepresented by the physical location and brand. For example, the sourcesmay include social networks (e.g., Facebook, Google+, Twitter), onlineforums, location-based sites (e.g., Foursquare, Yelp), and so forth.

Continuing in block 220, the system selects a first source from the listof sources from which to collect updated signal data. On subsequentiterations, the system selects the next source in the list of sourcesuntil all sources have been processed. The system may perform thisprocess on a periodic basis (e.g., daily) for each brand and physicallocation tracked. In some embodiments, the system may combine effort forgathering data from each source where a source contains data formultiple physical locations and/or brands. The system may also adhere toconfigured thresholds for acquiring data based on terms of use or otheracceptable use policies of each source.

Continuing in block 230, the system acquires signal data from theselected source. The signal data may include user posts, user activities(e.g., likes, mentions), or other user activities, that provideinformation related to the users' opinions or experiences with respectto the physical location and/or brand. The system may acquire data byaccessing programmatic APIs of each source, accessing an exposed webservice, crawling a website and scraping data, or other data collectionmethod. The system acquires the data and stores the data in a monitoringdata store for analysis to derive metrics for the business.

Continuing in block 240, the system analyzes the acquired signal data toidentify information related to the particular physical location andbrand. The system may perform a variety of types of content, semantic,demographic, and other analysis to determine the content and nature ofeach acquired content item. Information analyzed may include an authorof each content item, whether the content item is overall favorable orunfavorable, products mentioned in the content item, a message orsentiment conveyed by the content item, and so forth.

Continuing in decision block 250, if there are more sources in the listof sources, then the system loops to block 220 to select the nextsource. Although shown serially for ease of explanation, those ofordinary skill in the art will recognize that the system may processsources in parallel, acquiring signal data from each or from multiplesources at the same time for increased efficiency.

Continuing in block 260, the system aggregates signal data acquired fromthe sources to identify cross-source trends. The system rolls up datafrom each source to produce overall metrics for benchmarks such asnumber of total posts, number of positive mentions, number of negativementions, change in statistics from a prior period (e.g., drop/rise inday-over-day opinion), and so forth. By looking at multiple sources, thesystem paints a more complete picture of how people congregating onlineview an offline, physical business location.

Continuing in block 270, the system builds one or more benchmarks thattrack brand information by comparing historical information to theacquired and aggregated signal data. The system determines comparablelocations and segments to determine which historical data to compare.Comparisons may occur between locations of the same brand, betweenbrands of similar types (e.g., coffee shops), and so on. The system mayinclude a variety of explicitly and implicitly defined benchmarks. Insome embodiments, the system implicitly builds benchmarks by identifyingsimilar data across brands and locations and identifying past matchingbrands and/or locations as similar for comparison. Benchmarks may useacquired data directly as well as derivative data calculated upon theacquired data.

Continuing in block 280, the system detects one or more variances in theacquired and aggregated data compared with the historical information todetermine whether one or more triggers have been met for notifying auser. A variance may include any deviation from an expected value forany tracked metrics, including both explicitly and implicitly definedvariances. Customers of the system may define their own trigger pointsat which to be notified, and the system may define default triggerpoints based on historical information and observing values that are outof character or represent a significant change.

Continuing in decision block 290, if the system detects that a triggercondition has been met, then the system continues at block 295, else thesystem completes. Whether the system notifies a user or not, theprocessing to detect variances, build benchmarks, aggregate data, andacquire signals is stored in a data store so that the resulting data canbe compared with that of subsequent and later periods.

Continuing in block 295, the system notifies a user that at least onemetric satisfies a trigger. The notification may include any type ofcommunication, such as a text message, instant message, email, phonecall, push notification, or other communication. The notification may beaddressed to any user that has set up notifications, such as a managerof a retail location, a competitor, or others. The notification allowsthe recipient user to act upon the information, either by changingmarketing or taking other actions (e.g., increasing employee training,offering a new product, and so forth). After block 295, these stepsconclude.

FIG. 3 is a flow diagram that illustrates processing of the businessmonitoring system to receive a request to monitor a brand's performanceat a physical location through online events, in one embodiment.

Beginning in block 310, the system receives a monitoring request from arequesting user, wherein the monitoring request specifies one or morebrands and physical locations for which to monitor online activity toinfer offline behavior. The system may receive the request through aweb-based user interface that brand owners, competitors, serviceproviders, or other users access to submit requests for particular datafrom the system. The system may collect additional information andcreate a stored user profile for the requesting users so that therequesting user can access the site on an ongoing basis to receivereports and other forms of brand metric output.

Continuing in block 320, the system receives an identification of atleast one physical site for which the requesting users wants to collectmetrics. The site identification may include an address, name,latitude/longitude values, or other identifier that distinguishes aparticular location of a business. The system separate tracks eachphysical location of a business and may receive multiple siteidentifications that can be tracked so that metrics for the locationscan be compared with each other and with competing locations.

Continuing in block 330, the system receives a selection of one or moresources to track for online activity related to the identified physicalsite. The system may receive a set of sources identified by therequesting user and/or may automatically determine one or more sourcesto monitor, based on configuration from an administrator, previouslyidentified online sources, or other method. The system selects sourcesthat are likely to capture a complete and useful impression of how wellthe business is performing at its offline locations based on activitybetween users of online sources.

Continuing in block 340, the system defines one or more benchmarks basedupon which to monitor the selected sources for activity. The benchmarksmay be received from the requesting user and specify particular dataand/or inquiries that the user expects to be able to satisfy from themonitored sources. The benchmarks may include data directly availablefrom sources as well as derivative data. In some cases, benchmarks aredefined as comparisons between two data sets collected at differenttimes, such as records for the same brand and location from today andthe same day last year, today and yesterday, today and this day lastmonth, and so forth.

Continuing in block 350, the system sets one or more triggers based on athreshold received from the requesting user that indicates at least onebenchmark value for which the requesting user requests to be notified. Atrigger specifies a point in the data beyond which the requesting userwants to be informed of changes in the data so that the user can reactappropriately. For example, if online sentiment expressed towards alocation becomes sharply negative in a short period, the requesting usermay want to know. Likewise, if after a costly advertising campaign alocation experiences a rapid increase in foot traffic or other metrics,the user may want to be notified. Triggers allow users of the system toset these points of interest.

Continuing in block 360, the system stores a monitoring definition thatincludes the received brand, physical location, sources, benchmarks, andtriggers. The system stores this information in a data store accessed byprocesses for acquiring data from the sources, analyzing the data, andnotifying requesting users of any satisfied trigger conditions. Afterblock 360, these steps conclude.

The following figures illustrate example user interfaces produced by thebusiness monitoring system, in one embodiment.

FIG. 4 is a display diagram that illustrates a metric reportingdashboard of the business monitoring system, in one embodiment. Thedashboard 400 includes various widgets that display to a user of thesystem collected metrics for a business. For example, there are widgetsfor showing custom location segments and views 405, consumer sentimentscoring 410, most active online communities 415, historical sentimenttrends 420, by location trends 425, smart consumer profiles 430, reportsharing settings 435, social statistics for all locations 440, bylocation statistics 445, integrated maps 450, location trending 455, andlocation drill down information 460. From each of these, the user canselect and drill down to further understand or evaluate the underlyingdata.

FIG. 5 is a display diagram that illustrates four widgets for displayingbusiness metrics, in one embodiment. The widgets may be displayed in adashboard like that shown with reference to FIG. 4 or on separate detailpages. The diagram includes a first widget 510 that displays informationto a regional manager for his entire region. The first widget 510summarizes various statistics for the manager's region, such ascommunity size and recent increase in that size. The second widget 520displays locations monitored for the district manager, sorted bymessages detected at various online sources. The manager can use this tosee which locations are receiving the most online buzz or otheractivity. A third widget 530 shows the most common positive feedbacktopics for the monitored locations. A fourth widget 540 breaks thereported community size up by state.

FIG. 6 is a display diagram that illustrates two more widgets fordisplaying business metrics, in one embodiment. The diagram includes afifth widget 610 that displays ages of community members displayed as apie graph. The system gathers this information when the online signalsare acquired, from demographic data provided by a particular source, orinferred from other information. The diagram also includes a sixthwidget 620 that displays regions that have received the most negativefeedback. This may help a district manager or other executive toidentify trouble spots so that action can be taken to address anyproblems.

FIG. 7 is a display diagram that shows a map-based display of metrics,in one embodiment. The display 710 includes a map 720 followed by a listof locations 730 sorted by rank. In some embodiments, the businessmonitoring system calculates a VenueRank that factors in variousacquired metrics such as consumer sentiment, community growth/size,activity/engagement, a factor of how easily a business can be foundonline, and potentially other information to arrive at a relativeranking between business locations. In some cases, the VenueRank isbased on positive reviews and is time based so that older reviews rolloff over time. The VenueRank thus reflects those business locations thatare most highly regarded, and can be divided by various businesssegments, geographical regions, and so forth. By sorting the list oflocations 730, a user can quickly identify those business locations thatare highly regarded for the area displayed within the map 720.

From the foregoing, it will be appreciated that specific embodiments ofthe business monitoring system have been described herein for purposesof illustration, but that various modifications may be made withoutdeviating from the spirit and scope of the invention. Accordingly, theinvention is not limited except as by the appended claims.

1.-20. (canceled)
 21. A method for monitoring an effectiveness of amarketing campaign, the method comprising: receiving an indication of aparticular merchant location, the particular merchant location locatedwithin a first geographic region; identifying one or more other merchantlocations, each of the one or more other merchant locations each locatedwithin a different geographic region; receiving an indication of a timeat which the marketing campaign starts, the marketing campaign launchedwithin the first geographic region; identifying online signalsindicative of each of one or more specific metrics related to theparticular merchant location from the first geographic region and onlinesignals indicative of the one or more specific metrics related to thegeographic regions of the one or more other merchant locations;comparing the online signals from a first time period, indicative aperiod of time before the marketing campaign starts, to a second timeperiod, indicative of a period of time after the marketing campaignstarts; determining a change in the one or more metrics for theparticular merchant location; determining a change in the one or moremetrics for each of the one or more other merchant locations;determining a difference in between the change in the one or moremetrics for the particular merchant location and the change in the oneor more metrics for each of the one or more other merchant locations;and correlating the difference to the offline a marketing campaign. 22.The method of claim 21, further comprising: comparing a change in avalue of the one or more metrics for the particular merchant location tovalues of one or more historic benchmark metrics.
 23. The method ofclaim 21, further comprising: comparing a change in a value of the oneor more metrics for the particular merchant location to changes invalues of one or more concurrent benchmark metrics, the concurrentbenchmark metrics identified in a geographic region running the samemarketing campaign.
 24. The method of claim 21, further comprising:identifying a trend in the one or more specific metrics from the firsttime period to the second time period for the first geographic regionand the geographic regions of the one or more other merchant locations.25. The method of claim 21, wherein the one or more specific metricsinclude at least one of new engagements or total activity
 26. The methodof claim 21, wherein the first time period comprises a plurality of timeperiods before the start of the marketing campaign and the second timeperiod comprises a plurality of time periods after the start of themarketing campaign.
 27. The method of claim 26, further comprising:identifying a trend in the one or more specific metrics from theplurality of time periods before the start of the marketing campaign tothe plurality of time periods after the start of the marketing campaign.28. An apparatus for monitoring an effectiveness of a marketingcampaign, the apparatus comprising at least one processor and at leastone memory including computer program code, the at least one memory andthe computer program code configured to, with the processor, cause theapparatus to at least: receive an indication of a particular merchantlocation, the particular merchant location located within a firstgeographic region; identify one or more other merchant locations, eachof the one or more other merchant locations each located within adifferent geographic region; receive an indication of a time at whichthe marketing campaign starts, the marketing campaign launched withinthe first geographic region; identify online signals indicative of eachof one or more specific metrics related to the particular merchantlocation from the first geographic region and online signals indicativeof the one or more specific metrics related to the geographic regions ofthe one or more other merchant locations; compare the online signalsfrom a first time period, indicative a period of time before themarketing campaign starts, to a second time period, indicative of aperiod of time after the marketing campaign starts; determine a changein the one or more metrics for the particular merchant location;determine a change in the one or more metrics for each of the one ormore other merchant locations; determine a difference in between thechange in the one or more metrics for the particular merchant locationand the change in the one or more metrics for each of the one or moreother merchant locations; and correlate the difference to the offline amarketing campaign.
 29. The apparatus of claim 28, wherein the at leastone memory and the computer program code are further configured to, withthe processor, cause the apparatus to: compare a change in a value ofthe one or more metrics for the particular merchant location to valuesof one or more historic benchmark metrics.
 30. The apparatus of claim28, wherein the at least one memory and the computer program code arefurther configured to, with the processor, cause the apparatus to:compare a change in a value of the one or more metrics for theparticular merchant location to changes in values of one or moreconcurrent benchmark metrics, the concurrent benchmark metricsidentified in a geographic region running the same marketing campaign.31. The apparatus of claim 28, wherein the at least one memory and thecomputer program code are further configured to, with the processor,cause the apparatus to: identify a trend in the one or more specificmetrics from the first time period to the second time period for thefirst geographic region and the geographic regions of the one or moreother merchant locations.
 32. The apparatus of claim 28, wherein the oneor more specific metrics include at least one of new engagements ortotal activity
 33. The apparatus of claim 28, wherein the first timeperiod comprises a plurality of time periods before the start of themarketing campaign and the second time period comprises a plurality oftime periods after the start of the marketing campaign.
 34. Theapparatus of claim 33, wherein the at least one memory and the computerprogram code are further configured to, with the processor, cause theapparatus to: identify a trend in the one or more specific metrics fromthe plurality of time periods before the start of the marketing campaignto the plurality of time periods after the start of the marketingcampaign.
 35. A computer program product for monitoring an effectivenessof a marketing campaign, the computer program product comprising atleast one non-transitory computer-readable storage medium havingcomputer-executable program code instructions stored therein, thecomputer-executable program code instructions comprising program codeinstructions for: receiving an indication of a particular merchantlocation, the particular merchant location located within a firstgeographic region; identifying one or more other merchant locations,each of the one or more other merchant locations each located within adifferent geographic region; receiving an indication of a time at whichthe marketing campaign starts, the marketing campaign launched withinthe first geographic region; identifying online signals indicative ofeach of one or more specific metrics related to the particular merchantlocation from the first geographic region and online signals indicativeof the one or more specific metrics related to the geographic regions ofthe one or more other merchant locations; comparing the online signalsfrom a first time period, indicative a period of time before themarketing campaign starts, to a second time period, indicative of aperiod of time after the marketing campaign starts; determining a changein the one or more metrics for the particular merchant location;determining a change in the one or more metrics for each of the one ormore other merchant locations; determining a difference in between thechange in the one or more metrics for the particular merchant locationand the change in the one or more metrics for each of the one or moreother merchant locations; and correlating the difference to the offlinea marketing campaign.
 36. The computer program product of claim 35,wherein the computer-executable program code instructions furthercomprise program code instructions for: comparing a change in a value ofthe one or more metrics for the particular merchant location to valuesof one or more historic benchmark metrics.
 37. The computer programproduct of claim 35, wherein the computer-executable program codeinstructions further comprise program code instructions for: comparing achange in a value of the one or more metrics for the particular merchantlocation to changes in values of one or more concurrent benchmarkmetrics, the concurrent benchmark metrics identified in a geographicregion running the same marketing campaign.
 38. The computer programproduct of claim 35, wherein the computer-executable program codeinstructions further comprise program code instructions for: identifyinga trend in the one or more specific metrics from the first time periodto the second time period for the first geographic region and thegeographic regions of the one or more other merchant locations.
 39. Thecomputer program product of claim 35, wherein the one or more specificmetrics include at least one of new engagements or total activity 40.The computer program product of claim 35, wherein the first time periodcomprises a plurality of time periods before the start of the marketingcampaign and the second time period comprises a plurality of timeperiods after the start of the marketing campaign.
 41. The computerprogram product of claim 40, wherein the computer-executable programcode instructions further comprise program code instructions for:identifying a trend in the one or more specific metrics from theplurality of time periods before the start of the marketing campaign tothe plurality of time periods after the start of the marketing campaign.